Evaluating the quality of research co-production: Research Quality Plus for Co-Production (RQ + 4 Co-Pro).

BACKGROUND: Co-production is an umbrella term used to describe the process of generating knowledge through partnerships between researchers and those who will use or benefit from research. Multiple advantages of research co-production have been hypothesized, and in some cases documented, in both the academic and practice record. However, there are significant gaps in understanding how to evaluate the quality of co-production. This gap in rigorous evaluation undermines the potential of both co-production and co-producers. METHODS: This research tests the relevance and utility of a novel evaluation framework: Research Quality Plus for Co-Production (RQ + 4 Co-Pro). Following a co-production approach ourselves, our team collaborated to develop study objectives, questions, analysis, and results sharing strategies. We used a dyadic field-test design to execute RQ + 4 Co-Pro evaluations amongst 18 independently recruited subject matter experts. We used standardized reporting templates and qualitative interviews to collect data from field-test participants, and thematic assessment and deliberative dialogue for analysis. Main limitations include that field-test participation included only health research projects and health researchers and this will limit perspective included in the study, and, that our own co-production team does not include all potential perspectives that may add value to this work. RESULTS: The field test surfaced strong support for the relevance and utility of RQ + 4 Co-Pro as an evaluation approach and framework. Research participants shared opportunities for fine-tuning language and criteria within the prototype version, but also, for alternative uses and users of RQ + 4 Co-Pro. All research participants suggested RQ + 4 Co-Pro offered an opportunity for improving how co-production is evaluated and advanced. This facilitated our revision and publication herein of a field-tested RQ + 4 Co-Pro Framework and Assessment Instrument. CONCLUSION: Evaluation is necessary for understanding and improving co-production, and, for ensuring co-production delivers on its promise of better health.. RQ + 4 Co-Pro provides a practical evaluation approach and framework that we invite co-producers and stewards of co-production-including the funders, publishers, and universities who increasingly encourage socially relevant research-to study, adapt, and apply.

Evaluating research co-production: protocol for the Research Quality Plus for Co-Production (RQ+ 4 Co-Pro) framework.

BACKGROUND: Research co-production is an umbrella term used to describe research users and researchers working together to generate knowledge. Research co-production is used to create knowledge that is relevant to current challenges and to increase uptake of that knowledge into practice, programs, products, and/or policy. Yet, rigorous theories and methods to assess the quality of co-production are limited. Here we describe a framework for assessing the quality of research co-production-Research Quality Plus for Co-Production (RQ+ 4 Co-Pro)-and outline our field test of this approach. METHODS: Using a co-production approach, we aim to field test the relevance and utility of the RQ+ 4 Co-Pro framework. To do so, we will recruit participants who have led research co-production projects from the international Integrated Knowledge Translation Research Network. We aim to sample 16 to 20 co-production project leads, assign these participants to dyadic groups (8 to 10 dyads), train each participant in the RQ+ 4 Co-Pro framework using deliberative workshops and oversee a simulation assessment exercise using RQ+ 4 Co-Pro within dyadic groups. To study this experience, we use a qualitative design to collect participant demographic information and project demographic information and will use in-depth semi-structured interviews to collect data related to the experience each participant has using the RQ+ 4 Co-Pro framework. DISCUSSION: This study will yield knowledge about a new way to assess research co-production. Specifically, it will address the relevance and utility of using RQ+ 4 Co-Pro, a framework that includes context as an inseparable component of research, identifies dimensions of quality matched to the aims of co-production, and applies a systematic and transferable evaluative method for reaching conclusions. This is a needed area of innovation for research co-production to reach its full potential. The findings may benefit co-producers interested in understanding the quality of their work, but also other stewards of research co-production. Accordingly, we undertake this study as a co-production team representing multiple perspectives from across the research enterprise, such as funders, journal editors, university administrators, and government and health organization leaders.

Integrated Knowledge Translation Guiding Principles for Conducting and Disseminating Spinal Cord Injury Research in Partnership.

OBJECTIVE: To address a gap between spinal cord injury (SCI) research and practice by rigorously and systematically co-developing integrated knowledge translation (IKT) guiding principles for conducting and disseminating SCI research in partnership with research users. DESIGN: The process was guided by the internationally accepted The Appraisal of Guidelines for REsearch & Evaluation (AGREE) II Instrument for evaluating the development of clinical practice guidelines. SETTING: North American SCI research system (ie, SCI researchers, research users, funders). PARTICIPANTS: The multidisciplinary expert panel (n=17) and end users (n=35) included individuals from a North American partnership of SCI researchers, research users, and funders who have expertise in research partnerships. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Clarity, usefulness, and appropriateness of the principles. RESULTS: Data regarding 125 principles of partnered research were systematically collected from 4 sources (review of reviews, scoping review, interviews, Delphi consensus exercise). A multidisciplinary expert panel held a 2-day meeting to establish consensus, select guiding principles, and draft the guidance. The panel reached 100% consensus on the principles and guidance document. The final document includes a preamble, 8 guiding principles, and a glossary. Survey data showed that the principles and guidance document were perceived by potential end users as clear, useful, and appropriate. CONCLUSIONS: The IKT Guiding Principles represent the first rigorously co-developed, consensus-based guidance to support meaningful SCI research partnerships. The principles are a foundational tool with the potential to improve the relevance and impact of SCI research, mitigate tokenism, and advance the science of IKT.

Core knowledge translation competencies: a scoping review.

BACKGROUND: Knowledge translation (KT) is the broad range of activities aimed at supporting the use of research findings leading to evidence-based practice (EBP) and policy. Recommendations have been made that capacity building efforts be established to support individuals to enact KT. In this study, we summarized existing knowledge on KT competencies to provide a foundation for such capacity building efforts and to inform policy and research. Our research questions were "What are the core KT competencies needed in the health sector?" and "What are the interventions and strategies to teach and reinforce those competencies?" METHODS: We used a scoping review approach and an integrated KT process by involving an Advisory Group of diverse stakeholders. We searched seven health and interdisciplinary electronic databases and grey literature sources for materials published from 2003 to 2017 in English language only. Empirical and theoretical publications in health that examined KT competencies were retrieved, reviewed, and synthesized. RESULTS: Overall, 1171 publications were retrieved; 137 were fully reviewed; and 15 empirical and six conceptual academic, and 52 grey literature publications were included and synthesized in this scoping review. From both the academic and grey literature, we categorized 19 KT core competencies into knowledge, skills, or attitudes; and identified commonly used interventions and strategies to enhance KT competencies such as education, organizational support and hands-on training. CONCLUSIONS: These initial core KT competencies for individuals provide implications for education, policy, knowledge brokering, and future research, and on the need for future evaluation of the KT competencies presented. We also discuss the essential role of organizational support and culture for successful KT activities/practice.

Evaluation of the "Foundations in Knowledge Translation" training initiative: preparing end users to practice KT.

BACKGROUND: Current knowledge translation (KT) training initiatives are primarily focused on preparing researchers to conduct KT research rather than on teaching KT practice to end users. Furthermore, training initiatives that focus on KT practice have not been rigorously evaluated and have focused on assessing short-term outcomes and participant satisfaction only. Thus, there is a need for longitudinal training evaluations that assess the sustainability of training outcomes and contextual factors that may influence outcomes. METHODS: We evaluated the KT training initiative "Foundations in KT" using a mixed-methods longitudinal design. "Foundations in KT" provided training in KT practice and included three tailored in-person workshops, coaching, and an online platform for training materials and knowledge exchange. Two cohorts were included in the study (62 participants, including 46 "Foundations in KT" participants from 16 project teams and 16 decision-maker partners). Participants completed self-report questionnaires, focus groups, and interviews at baseline and at 6, 12, 18, and 24 months after the first workshop. RESULTS: Participant-level outcomes include survey results which indicated that participants' self-efficacy in evidence-based practice (F(1,8.9) = 23.7, p = 0.001, n = 45), KT activities (F(1,23.9) = 43.2, p < 0.001, n = 45), and using evidence to inform practice increased over time (F(1,11.0) = 6.0, p = 0.03, n = 45). Interviews and focus groups illustrated that participants' understanding of and confidence in using KT increased from baseline to 24 months after the workshop. Interviews and focus groups suggested that the training initiative helped participants achieve their KT project objectives, plan their projects, and solve problems over time. Contextual factors include teams with high self-reported organizational capacity and commitment to implement at the start of their project had buy-in from upper management that resulted in secured funding and resources for their project. Training initiative outcomes include participants who applied the KT knowledge and skills they learned to other projects by sharing their knowledge informally with coworkers. Sustained spread of KT practice was observed with five teams at 24 months. CONCLUSIONS: We completed a longitudinal evaluation of a KT training initiative. Positive participant outcomes were sustained until 24 months after the initial workshop. Given the emphasis on implementing evidence and the need to train implementers, these findings are promising for future KT training.

Protocol of a scoping review on knowledge translation competencies.

BACKGROUND: Knowledge translation (KT) activities can reduce the gap between "what is known" and "what is done". Several factors hinder or facilitate KT activities including individual characteristics and organizational attributes; we will focus on individual healthcare professional modifiable characteristics. The purpose of this scoping review is to summarize knowledge on KT competencies for knowledge users, knowledge brokers, and knowledge producers/researchers to support evidence-based practice (EBP) and inform policy and research in health. Our objectives are to explore the relevant theoretical and empirical literature; map the publications for key themes and research gaps of KT competencies, and interventions for enhancing KT competencies; summarize and disseminate findings; produce an action plan and research agenda; and develop self-assessment tools (the KT Pathways) for professional development for our three target audiences. METHODS: The scoping review method will guide our study by following six stages: formulating the research question; identifying relevant studies; selecting the literature; charting the data; collating, summarizing, and reporting the results; and developing a KT plan and consulting stakeholders involved in the fields of KT, EBP, evidence-informed policy-making, and/or research. We will include empirical and theoretical/conceptual peer-reviewed and grey literature in health that examine knowledge user, knowledge broker and knowledge producer KT competencies. Publications written in the English language and published after 2003 only will be considered. Our multidisciplinary research team will collaborate using technology (i.e., WebEx for discussions and a Web 2.0 website for storing documents). Our KT plan consists of an Advisory Group and dissemination plan of the findings. DISCUSSION: We expect the identified KT competencies to contribute to the KT science by providing positive outcomes in practice, policy, education, and future research. Incorporation of the core KT competencies may enhance safety, effectiveness of clinical care, and quality of health outcomes; contribute to and facilitate collaboration among practitioners, knowledge users, knowledge brokers, researchers, employers, and educators; improve education of healthcare professionals and inform policy-making process; benefit practitioners by guiding their KT professional development to become effective at moving evidence into practice and policy; guide suitable interventions and strategies to enhance KT activities in the health sector; and direct future research.

Reviewer training to assess knowledge translation in funding applications is long overdue.

BACKGROUND: Health research funding agencies are placing a growing focus on knowledge translation (KT) plans, also known as dissemination and implementation (D&I) plans, in grant applications to decrease the gap between what we know from research and what we do in practice, policy, and further research. Historically, review panels have focused on the scientific excellence of applications to determine which should be funded; however, relevance to societal health priorities, the facilitation of evidence-informed practice and policy, or realizing commercialization opportunities all require a different lens. DISCUSSION: While experts in their respective fields, grant reviewers may lack the competencies to rigorously assess the KT components of applications. Funders of health research-including health charities, non-profit agencies, governments, and foundations-have an obligation to ensure that these components of funding applications are as rigorously evaluated as the scientific components. In this paper, we discuss the need for a more rigorous evaluation of knowledge translation potential by review panels and propose how this may be addressed. CONCLUSION: We propose that reviewer training supported in various ways including guidelines and KT expertise on review panels and modalities such as online and face-to-face training will result in the rigorous assessment of all components of funding applications, thus increasing the relevance and use of funded research evidence. An unintended but highly welcome consequence of such training could be higher quality D&I or KT plans in subsequent funding applications from trained reviewers.

How funding agencies can support research use in healthcare: an online province-wide survey to determine knowledge translation training needs.

BACKGROUND: Health research funding agencies are increasingly promoting evidence use in health practice and policy. Building on work suggesting how agencies can support such knowledge translation (KT), this paper discusses an online survey to assess KT training needs of researchers and research users as part of a Canadian provincial capacity-building effort. METHODS: The survey comprised 24 multiple choice and open-ended questions including demographics, interest in learning KT skills, likelihood of participating in training, and barriers and facilitators to doing KT at work. More than 1,200 people completed the survey. The high number of responses is attributed to an engagement strategy involving partner organizations (health authorities, research institutes, universities) in survey development and distribution. SPSS was used to analyze quantitative results according to respondents' primary role, geographic region, and work setting. Qualitative results were analyzed in NVivo. RESULTS: Over 85 percent of respondents are interested in learning more about the top KT skills identified. Research producers have higher interest in disseminating research results; research users are more interested in the application of research results. About one-half of respondents require beginner-level training in KT skills; one-quarter need advanced training. Time and cost constraints are the biggest barriers to participating in KT training. More than one-half of respondents have no financial support for travel and almost one-half lack support for registration fees. Time is the biggest challenge to integrating KT into work. CONCLUSIONS: Online surveys are useful for determining knowledge translation training needs of researchers, research users and ultimately organizations. In this case, findings suggest the importance of considering all aspects of KT in training opportunities, while taking into account different stakeholder interests. Funders can play a role in developing new training opportunities as part of a broad effort, with partners, to build capacity for the use of health research evidence. Survey results would ideally be complemented with an objective needs assessment based on core competencies, and should be acted on in a way that acknowledges the complexity of knowledge translation in healthcare, existing training activities, and the expertise stakeholders already have but may not refer to as knowledge translation.

Translating evidence into practice: the role of health research funders.

BACKGROUND: A growing body of work on knowledge translation (KT) reveals significant gaps between what is known to improve health, and what is done to improve health. The literature and practice also suggest that KT has the potential to narrow those gaps, leading to more evidence-informed healthcare. In response, Canadian health research funders and agencies have made KT a priority. This article describes how one funding agency determined its KT role and in the process developed a model that other agencies could use when considering KT programs. DISCUSSION: While 'excellence' is an important criterion by which to evaluate and fund health research, it alone does not ensure relevance to societal health priorities. There is increased demand for return on investments in health research in the form of societal and health system benefits. Canadian health research funding agencies are responding to these demands by emphasizing relevance as a funding criterion and supporting researchers and research users to use the evidence generated.Based on recommendations from the literature, an environmental scan, broad circulation of an iterative discussion paper, and an expert working group process, our agency developed a plan to maximize our role in KT. Key to the process was development of a model comprising five key functional areas that together create the conditions for effective KT: advancing KT science; building KT capacity; managing KT projects; funding KT activities; and advocating for KT. Observations made during the planning process of relevance to the KT enterprise are: the importance of delineating KT and communications, and information and knowledge; determining responsibility for KT; supporting implementation and evaluation; and promoting the message that both research and KT take time to realize results. SUMMARY: Challenges exist in fulfilling expectations that research evidence results in beneficial impacts for society. However, health agencies are well placed to help maximize the use of evidence in health practice and policy. We propose five key functional areas of KT for health agencies, and encourage partnerships and discussion to advance the field.

Third generation anticonvulsants in bipolar disorder: a review of efficacy and summary of clinical recommendations.

BACKGROUND: To review the literature on efficacy of third generation anticonvulsants for treatment of bipolar disorder and provide clinical recommendations. METHOD: Open and controlled studies, case reports, and case series on the efficacy of lamotrigine, gabapentin, topiramate, tiagabine, and zonisamide were located through electronic searches of several databases, by manual search of proceedings of international meetings, and through contacting authors of recent reports. RESULTS: Lamotrigine is the best studied anticonvulsant and has efficacy in acute bipolar depression and in longer term treatment of bipolar depression as well as rapid-cycling bipolar II disorder but not in acute mania. Open reports suggest usefulness of gabapentin as an adjunct in bipolar disorder, but double-blind trials failed to confirm efficacy in acute mania and treatment-resistant rapid-cycling bipolar disorder. Topiramate is reported to be effective in acute mania and rapid-cycling bipolar disorder in several open studies, but methodological problems in a double-blind study led to a failed study in acute mania. However, topiramate may lead to weight loss in some patients. Zonisamide deserves further investigation, but tiagabine does not appear to be useful in acute mania. CONCLUSION: Lamotrigine clearly fills an unmet need in treating bipolar depression and rapid-cycling bipolar disorder. Other third generation anticonvulsants with the exception of tiagabine offer promise but require confirmation of their efficacy from double-blind studies.

PET study of [(18)F]6-fluoro-L-dopa uptake in neuroleptic- and mood-stabilizer-naive first-episode nonpsychotic mania: effects of treatment with divalproex sodium.

OBJECTIVE: Although dopaminergic hyperactivity has been implicated in mania, the precise location in the brain of the abnormality is unclear. This study assessed presynaptic dopamine function in neuroleptic- and mood-stabilizer-naive nonpsychotic first-episode manic patients before and after treatment with divalproex sodium by measuring [(18)F]6-fluoro-L-dopa ([(18)F]DOPA) uptake in the striatum with positron emission tomography (PET). METHOD: Thirteen patients with DSM-IV bipolar I disorder, manic episode, and 13 healthy comparison subjects underwent [(18)F]DOPA PET scans. Ten of the 13 patients had repeat PET scans 2-6 weeks after beginning treatment with divalproex sodium monotherapy. [(18)F]DOPA uptake rate constants (K(i) values) in the striatum were calculated by using graphical analysis with activity from the occipital cortex as the input function. RESULTS: No significant differences in [(18)F]DOPA uptake rate constants in the striatum were found between the manic patients and the comparison subjects. After treatment with divalproex sodium, [(18)F]DOPA rate constants were significantly reduced in the patients and were lower in the patients than in the comparison subjects. CONCLUSIONS: Although presynaptic dopamine function as reflected by [(18)F]DOPA uptake is not altered in mania, presynaptic dopamine function in manic patients was lower after treatment with divalproex sodium.